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Abstract 


A covputer-controlled talaoparator systes which is basad on task* 
rafarancad sansor*aidad control has baan davalopad to study suparviscry aani* 
pulation* This systast calla^ SUFBBHAlIt is capable of parforsiag coaplicatad 
tasks in raal*tiaa by utilising tha operator for high*laval functions related 
to tha unpredictable portions of a task, while tha subordinate aachine par* 
forms the more well*defined subtasks under human suparvison. 

To dateraina whether supervisory control schemes such as these offer any 
advantage over manual control under real-time conditionsy a number of experi- 
ments involving both simple and complicated tasks ware performed. Six repre- 
sentative tasks were chosen for the study: (1) obtaining a tool from a rack, 
(2) returning the tool to the rack* (3) removing a nut; (4) placing samples 
in a storage bint opening and closing a valvSt and (6) digging with a 
shovel. The experiments were performed under simulated conditions using four 
forms of manual control (i.e.t switch ratet joystick ratei master- slave posi- 
tion control t and master-slave with force feedback) t as well aa supervisory 
control. Through these experiments the effectiveness and quality of control 
were evaluated on the basis of the time required to complete each portion of 
the task and the type and number of errors which occurred. 

Even under the %est'* control conditions (i.e.« no degraded sensor or 
control loops due to time delaysy restricted bandwidthSy etc.) supervisory 
control was found to improve performance for all forms of manual control 
except force- re flee ting master-slave which was found to be slightly faster 
than supe^isory control « b rtore prone to errors. With degraded sensor or 
control loops it is fairly predictable that supervisory control will show 
even more advantage, through the latter experiments are yet to be done* 

1 . nmoDucTioM 

Teleoperators have traditionally relied on relatively simple and direct 
man-machine interfaces control. However, with the advent of microcom* 
puters and advanced sensor techniques it is now possible to design and build 
a hierarchical control system in which the operator is responsible for the 
higher-level functions related to the unpredictable portions of the task, 
while the subordinate machine performs the SK>re well-defined subtasks under 
human supervision. Control based on a supervisor-subordinate relationship 
such as this is called **supervisory control** general, the hwan 
operator coaminicates intermittently with the computer, and the computer, in 
turn and continuously in time, controls the sensors and actuators of the 
vehicle and manipulator. In essence, the teleoperator system acts as an 
autonomous '* robot'* for short periods while in the pursuit of task goals pre- 
viously programmed by the operator or updated on the last cycle of coaminica- 
tion. This mode of control promises more precision for certain tasks, less 
susceptibility to failure in the event of communication channel breakdown, 
and greater efficiency than direct human control. 
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to invest ig«to the relative aerits of supervisory control applied to 
teleopera tors t a task^referenced sensor-aided supervisory systesf called 
SUPERMAN > vas built and experiaents vere perforaed. Thia paper will evaluate 
those experiaents and b^ coaparison of various conventional control aodes 
with supervisory control y deaonstrate that supervisory aanipulatlon does 
iaprove perforaance in the aajority of cases* 

2* METHOD 


Apparat us 

The aajor eleaents of the SUPERMAN systea are a aodified Argonne E2 
aaster-slave aanipulator with six degrees-of-freedoay a dedicated control 
interface (DASI)y and an Interdata 70 cosiputer. Designed for efficient aan-* 
aachioc interaction with both analog and syabolic control input Sy the systea 
can be coManded by a Variety of conventional control aodes as well as super- 
visory* In additiony tiae delay and/or noise can be added for experiaental 
purposes* 

Using both analog and syabolic cosaandsy a aanipulatlon can be taught 
and/or deaonstrated to the coaputer* Trained aanipulations can he trans- 
foraed froa one coordinate systea to another so that once the generic charac- 
teristics of a task have been learned y the aachine can perfora siailar tasks 
in different locations without further training* When the huaan operator 
requires a particular trained aanipulation be siaply ** initialises'* the new 
coordinate systea relative to the old by aoving the teleoperator hand to the 
starting point of the task (e*g*y grasping a nut or valve handle) and signals 
for execution* Certain objects in the task environaent cany of course y aain- 
tain their original coordinates. For a coaplete developaent of task trans- 
fomacions related to supervisory control see refs. 2 and 3* 

Since the aanipulator can sense the forces generated during the task, 
supervisory progrsas can call for repeated aoveaents which y upon certain 
touch conditions becoaing trusy branch into other aovaients* For exaapley 
repeated hand aoveaents can grasp a nuty unscrew it by one revolutiony pull 
back to test whether it is off andy if it iSy place it in a bucket ory if it 
is not I repeat the operation* Siailar supervisory progrsas have been applied 
to attaching a nut to a bolty opening and closing a valve y scooping dirt and 
so on. Further information on the SUPERMAN systea can be found in ^ef. 3* 

&ne aanipulator laboratory was arranged as shown in fig* 1 during the 
experiaents* To siaulace reaote conditions the operator viewed the task 
enviroiaient through either a mono or 2-view television systea* The video 
system consisted of two black and white high-resolution 9 in* monitors y a 
fixed camera with wide angle lensy and a sooa cmmrr with pan 4 tilt* 

Figure 2 shows the aanipulator environaent and the experiaental tasks 
designed for this study* The tool rack and sample buckets remained in the 
locations shown throughout the experiaents since these pieces of equipment 
are usually rigidly attached to the teleoperator vehicle in real applica- 
tions. Also shown in the figure are the movable task hub and task board on 
which representative tasks such as valves y bolts y etc were mounted* The 
location of the task hub and board were changed throughout the study to simu- 
late the random task/vehicle relationships which are typical of the arbitrary 
environaents found in narine and space applications* 
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Kxpri— nf I D^iign 

Six bxiic tasks vara idantifiad for axpariaantal invast igat ion: ^ (1) 

tool ratriavsl; (2) tool ratum; (3) taking a nut off; (4) grasping an objact 
and placing it in a containar; (5) opaning/closing a valva; aad (6) digging. 
In addition, four annual control sodas vara dalinaatad as iaportant axpari* 
aantal paraaatat': (1) svitch fixad rata; (2) Joystick variabla raca; (3) 

aastar-slav^ ition control; and (4) aastar^alava position control vith 
fores raflaction. Vith ragard to tha vidao arraagaaant. both aono and 2-viav 
conditions vats tastad for coaparison, Dua to tiaa constraints only thraa 
subjects vara usad for four of tha tasks (tool ratriavalf tool raturOt nut* 
off and saaplsr), and only ona subjact vas usad for tha raaaining tvo (opan/ 
doss valva and diggar). Each axpariaant vas parforaad 3 tiaas by aach sub* 
jset to obtain a statistical as# and standard daviation* Both annual and 
suparvisory control vara usad. 

Thass conditions rasult in a total of 1120 axpariaantal runs. Sinca 
this vould raquira an inordinata saKMint of tiaa, tha axpariaantal load vas 
rsducsd CO 6B0 runs by noting that soaa of tha tasks, or portions '^f tha 
tasks, had constant coaputsr sxacution tiaas (saa raf. 3 for datails). 

Subjects and Trsinina 

Three classes of subjects vara usad for thasa expariaants, ona expari* 
enced, four vail trained, and tvo untrained subjacts. 

Tha vail trained subjacts had an avaraga of 20 hours training given in 
13 ainuta intervals for aach of tha control aodas. Canarally, after tha sub* 
jacts practiced for 13 ainutas vith a particular control aoda a siaulatad 
task vas parforaad. Whan the subjacts appeared to show a plateau, expari* 
aenls wra begun. Sinca Che expariaants usually stretched over a period of 
several days, the subjects vara asked to **raparfora*' soaa of tha tasks dua to 
a **aistake". If tha subjects showed aarkad iaprovaaant the tasks vara par* 
foraed again until the learning curve lavallad off, Tha four trained sub* 
jeeCs vare given incentives to perform veil in the fens of bonuses which 
vould be avarded to tha bast coabinad time and error rates in any control 
category. 

Tha first author was usad as the baseline experienced subject. Vith 
over 200 hours of practice on aanipulator systaas and intlmata knowladga of 
the SUFBEM4II syttaa, it may be reasonably assiaad that tha axpariancad sub* 
ject underwent litrla or no learning. Tha axpariancad subjact performed all 
of tha tasktt without a **varmrup" period. 

The untrain^ subjects had a total of 3 hours training time for all con* 
trol modes (i.e., 30 minutes par control mode and viewing condition). The 
learning curves of the untrained subjacts vara not observed. The only 
requirement placed on their training sessions was to insure that aach control 
nod# was given equal training time. After tha 3 ona*hour fsmiliaric*' and 


^Although it may appear that the tool retrieval and return tasks are simply 
the reverse procedure of one another, these tasks do have fundamentally 
different raquiramants. To clarifv, consider that tha retrieval task 
required the subjects to locate a lit x 3/4 inch t*^l handle vith tha end 
effector docking plate vhile the return taek required the subjects to mate 
tvo 1/8 inch pins and holes. 
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•djustse^t periods were ovrr cn« subjects wsrs sllovsd 7 ^ hours of rest 
and Che expevismts were begun. 

Procedure 

The experiments were scored on the basis of recorded time end errors. 
The subjects were not given specific instr\%ctioos to minimise either ^ality^ 
but only to weigh them equally. Each subject wast however, given a criterion 
by which successful completion of the cask would be measured (these criteria 
will be specified on the following pages). The experiments were not redone 
when errors occurred, regardless of the magnitude, unless it was impossible 
to proceed with the task (e.g., a collision with an object that blew a ruse, 
etc*)* The tasks were randomised whenever possible to negate the effects of 
variables which the experimenters were not aware of (e.g*, particularly easy 
or difficult taek positions, short term learning effects, etc.). All tasks 
started from a prespecified position so that comparisons of supervisory 
initialisation times could be made across control modes. 

The procedure for each of the representative tasks was as follows: 

a) Tool-Eetrleval Task - The first task required the subject to start 
with the end effector positioned near the task hub. On the experi^ 
menter*s signal, the subject moved the end effector tc the tool 
rack, obtained the tool, being sure it was properly seated in the 
hand, and returned *ith the tccl to the starting postion. The sub* 
jects were told that the success or failure of the task was measured 
by whether a scHd connection between the tool handle and end effec* 
ter was achieved. Execution of this task under supervisory control 
simply involved a button push. 

b) Tool*Retum Task - For the second task the subject started from a 
position next to the task hub with the tool in hand, and on the 
experimenter’s signal, sK>ved to the rack, replaced the tool insuring 
that it was properly seated, and returned to the initial position* 
The operators were told that the success or failure of the task was 
determined by whether or not the tool was properly replaced on the 
rack* To properly seat the tool on the rack required that both of 
the 1/8 inch rack pins were engaged in the handle and that the tool 
was completely pushed onto the pins. This task was executed under 
supervisory control through a simple button push. 

c) Nut*Removal Task - This experiment began with the end effector posi- 
tioned over the valve on the task hub* On the experimenter's sig- 
nal, the subject moved the end effector from the valve to the nut, 
oriented the hand, and removed the nut. The general procedure used 
by the subjects and computer was to turn 180^, pull back to test 
if the nut was off, and then aither revaraa 180^ and continue, or 
remove the nut. ^rior to the tass, the operators were told that tha 
task would be considered successfully completed if the nut could be 
removed without losing it. Under supervisory control ihe operator 
initial ixed the task by moving from the starting poaition to the 
nut, orienting the hand with the rotational axis of the nut, and 
signaling the computer to remove it. 

d) Sampling Task - The fourth task required the subject to pick-up 
thirteen randomly placed samples and put them in one of two buckets 
according to their sice. The subjects were told thsv their success 


597 



CRlGiWAL PAGE 13 

OF POOR QUALITY 


or failure to cooploto tho task would ba aaaaurad by bow May aai^ 
plus wavu auccaaafully placad in tba propar buekata* Uodar aupar* 
viaory control tba oparator initial iaad tba task by placing tba and 
atf actor ovar tba aanpla and signaling tba conputar to placa it in 
tha appropriata buckat* Tba conputar ratumad control to tba tub* 
jact at tba location warn tha aanpla was graapad. tba oparator than 
novad to anotbar aanpla, initialitad, and continuad until all 13 
aanplaa wars in rba buckata* 

a) Opan/Cloaa Valaa Taat - Tbia axparinant ra^irad tba aubjact to 
position tba and af factor ovar tba nut on tba task bub, and than, on 
tba axparinnntar'a signal, tba aubjact noaad to tba aalaa, oriantad 
tba hand, and opanad or cloaad tba aalua an raguirad (opaning and 
closing tasks wars switcbad aftar sack axparinant)* Tba aubjact van 
raquirad to continua until tba aalaa oparation was conplata. To 
initial isa this task undar suparaisory control tba oparator oriantad 
tha and af factor on tba rotational axis of tba aalaa and signalad 
tha conputar aitbar to opan or closa it as raguirad* Tba conputar 
cbackad tba rotational torquas to datamina if tba task bad baan 
conplatad. 

f) Di ttina Task - Tba final task raquirad tba subjact to rnnous a spa* 
cifiad nnount cf soil fron a box by filling a buckat with a sboral* 
This task is conposad of a nunbar of subtaskst (1) tba aboval is 
positionad to ranora tha soil, (2) tha shoral is pusbad into tha 
soil and liftad out, and (3) tha soil is transportad to tba buckat 
and droppad in* Tba subjact was taquirad to continua until tba 
buckat was fillad. Dndar suparwisory control tba positioning of tha 
shoval was parfomad nanually (i.a*, tba oparator dacidad whan and 
wham to dig) whila tha scooping and dropping actions wars axacutad 
by tha conputar* 


3 . nSDLTS 

It has baan shown bv a nunbar of invastigators that tha tins mquirad to 
parfom a task can ba attributad to a nunbar of distinctly diffamnt 
notions. For axanpla, ona classification dividas tha task tins for control 
with a tins dalay into savants mlatad to Mt, transport, and position 
notfirns [^3* * pag*in*tha*hola task Hill ^53 has shown that tbam am 

two indapandant notions which dataraina tba tot^ task tins undar nanual con* 
trol - gross traval and pmcision. This papar will usa a sinilar schaan to 
dascriba tha task conplation tins for a suparvisory systant 

'tt ■ ‘t ♦ S 

wham • 

t_ • Task Tina 
TT 

• Tina raquirad by tha hunan oparator to initialiia tha task. 
This tins is priMrily a function of tha initial hand f task Iocs* 
tions and tha nanual control noda usad to locata tha task. 

tp ■ Tina raquimd by tha conputar to par Torn tha task. This tins is 
prinarily a function of tha task conpTaxlty. 

Tha datamination of thasa tinas it rathar sinpla dua to tha discontinuity in 
control which occurs during tha trada fron nanual initialisation to conputar 
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execuCioo (this '^discontinuity*' is a desired result since trading of control 
should be ''apparent** ^3>63)* 

Figures 3-6 are plots of typical data (see legend belov for figure 
abbreviations). The data recorded during the supervisory experiments have 
been divided into initialization and performance times to indicate the time 
spent by each action. Each of the time bars is the result of oata averaged 
over tvo trained subject s, except for fig. 6 which is averaged over three 
trained subjects. The lines to the left of the manual control birs give the 
range over which the trained subjects performed the task. Fo^ comparison, 
the average time for an inexperienced subject to perform the first three 
tasks is also given (denoted by triangles). The mean times of the untraii^ 
subjects were always above the maximum value of the trained s objects for the 
same task and control mode. The lower portion of each figure (figs. 3b-6b) 
plots the mean number of errors whici occurred under manual and supervisory 
control (for a specific breakdown of the individual errors see ref. 3). 

LEGEND; Key to Abbreviations Used in Text 

MS - Master-slave with force feedback 
MS NO FTS - Master -slave without force feedback 
JVkC - Joystick variable rate control 
SVRC - Switch fixed/variable rate control 

4. EVALUATION 


Manual Control 

Predictably, the task completion time increased with control complexity 
for all tasks. Viewing conditions (mono and 2-view) appeared to affect tasks 
which required precision movements (e.g. , return tool and nut-off), but had 
little or no effect on the less precise tasks (e.g., sampling). In general, 
the number of ervsjrs increased as the control complexity increased from 
master-slave tc switch rate. However, for some of the tasks a sharp decrease 
in errors was noticed between joystick and switch rate control (e.g., see 
figs. 5b and 6b). This effect is attributable to t«io factors: (1) the 

increased attention and care each operator exhibited during switch rate con- 
trol modes (i.e, to move from point A to point B requires considerable 
thought and effort with switch rate control, ^t under joystick control the 
desired movement only requires a push on the stick), and (2) the coincidental 
matching of the task degrees of freedom and control degrees of freedom (e.g., 
in the valve or nut-off tasks the axis of rotation corresponded with the hand 
axis of rotation). 

Table I gives the ratio of task completion times for each control mode 
with respect to the "best” control case, master-slave with force feedback. 
The ratios are given for each subject, task and viewing condition. The 
untrained subjects are denoted by U1 and U2, the trained subjects are denoted 
by Tl, T2, T3 and T4, and the experienced subject is denoted by £1. The 

table shows a niaber of interesting trends: (1) the ratios increase with 

increasing control complexity, (2) the ratios are approximately constant 
across subjects (both trained and untrained) within a given task, (3) the 
ratios are constant across viewing conditions, and (4) the ratios are not 
constant across tasks (the tasks have been arranged in the table so that the 
ratio increases as the page is read from top to bottom). A number of other 
irvestigators have found similar trends t^,8,9,103. 
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Fig. 3«: Av«r«f€ Tool*lt#cri«v«l TIm. Each har givat cha avaraga tiaa 

of tao Aubjacca. Tha A a^fvbol rapraaanta cha Man ciaa for 
an untrainad anbjact. Tha cappad liaaa show cha Cocal ranga 
of daca for cha crainad aubjacCs. 



Fig. 3b: Expaccad Eisbar of Too 1-Racri aval Errora. Each data poinc 

rapraaanca cha avaraga arror raca of Cwo Crainad aubjacCa. 
Poaaibla arrora includad colliaiona, dropping cha cool, 
and not aaacing cha handla in Cha and affactor proparly. 
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ig. 4«: Average Tool-EeCum Tiae* Each bar repreaenta the average 

tiM of Cifo trained aubjecta and each A givea the aean 
tiae for an untrained aubject* The capped linea repreaent 
the total range of data for the trained aubjecta. 



Fig. 4b: Expected Nuaber of Tool-Return Error a. Each data point 

.epreaenta the average error rate of tvo trained aubjecta. 
Poaaible errora included colliaiona, dropping the tool, 
and not aeating the handle on the rack properly. 
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ig. 5«: Av«rag« Hut-E«MV<1 TiM. B«ch bar rapraaancs Cha avaraga 

tiaa of two trainad aubjacta and aach A givaa tha aaan 
tiaa for an untrainad aubjact. Tha cappad linaa rapraaant 
cha total ranga of data for tha trainad aubjact. 



Fig. 5b: Expactad Niabar of Rut^Ranoval Brrora. Each data point 

rapraaanta tha avaraga arror rata of two trainad 
aubjacta. Poaaibla arrora includad colliaiona and 
droppir^ tha nut. 
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MS MS jvm: svnc 

NOF^B 


MS MS JVfK SVIK 
NOFTB 


ExpBctfld Ntal>«r of Scapliug Errors. E«ch d«ts point 
roprssBiits th« avBrBgo trror rat# of thraa trainad 
aubjacta for 13 aaKpling actions • Poaaibla arrora 
includad collisiona, taiaaad buckata» lost asMplaa, 
and (undar auparviaory control) praaaing tha wrong 
button. 
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T«bl€ I: Ratio of TIm to Perfora Task Undar Givan Control Moda to TIm 

to Parfom Task Undar Mastar-Slaaa with Forca Faadback (GM/HS). 
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Suparviaory Control 

Aa ifould ba axpactad^ tha tiaa ra<|uirad by tha coaputar to parform ita 
portion of tha tank rasainad fizad ragardlaaa of tha aannal control soda froa 
which tha hiaan oparator iaauad tha axacution coMand* Alto, ainca tha only 
action raquirad of tha oparator to initiata tha tool*ratriayal and ratum 

taaka waa a button puah, fha abaanca of initialisation tiMa in figa. 3a and 
4a vaa not aurpriaing. Tna ranaining taaka , including thoaa not ahown in 

thia papar, had initialisation tinaa aajociatad with tha oycrall taak tiaa* 
Aa aaan in figa. 5 and 6 tha initialisation tinaa incraaaad with control 

coBplaxity* 

^4bla 2 giyaa tha ratios of tha taak conplation tinaa undar nanual 

conIro^ to tbs tinaa undar aupanriaory control* The ratios ara giran for 
each MUthjtcC , taak and yiawing condition* Tha ratios ralatiya to computer 
control (Tab*.it 2) do not show tha sana trends aa thoaa ralatfya to aaatav- 
alavtt cont^o^ (Table 1)* It is intaraating to note that in contrast to tha 
coraiatant ratios of Table It the coaputar control ratios of tha untrained 
subJiiCts sfa significantly higher than tha trained subjactss claarlyt 
untrained aubjacta ^ain aora froa suparyiaory control than trained subjects • 
Gains froa suparviaory control for any manual node ara seen to ba most aigni*- 
ficant for tasks which do not raquirs initialisation procedures other than a 
button push (i*a*t tool*ratrisyal and tool*ratum)* The control aoda coluans 
clearly indicate tha results of tha SUPRSMAR axpariaantsi (1) aastar-slaya 
with force faadback rarely benefits froa supervisory control, (2) aastar* 
slave without forca faadback can profit froa aupanriaory control in tasks 
which require forca feedback, and (3) both forms of rata. control can ba aided 
by aupanriaory routines ragardlaaa of tha task* 
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Tftbltt 2: Ratio of Tisa to Perfors Task Undar Hanual Control to Tiaa to 

Parfora Task Undar Suparviaory Control (HC/SC). 
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In all caaas the error rates for supervisory control vara less than aan- 
ual control. Hovevert an interesting error was noted during the saapling 
expariaants * occassionally the subjects pressed an incorrect button sending 
the saaple to the vrong bucket. 


5. DISCUSSION 

Theoretically there is no reason vhy aaster-slave vith force feedback 
should be any faster than supervisory control. Consider that the coaputer 
could siaply aiaic the hiaan operator's bast tiajectory^ and hencey be at 
least as fast. Unfortunately 9 in practice thar* is always a certain overhead 
associated with re transformation of coordinates , trajectory calculations and 
sensor logic. Also 9 it was generally observed that the subjects were making 
adaptive, orchestrated motions, whereas the computer was limited to more 
rigidly defined trajectories and states. In light of these observations it 
can be said that the faster master-slave times make more of a statement about 
the direction that future studies dealing with supervisory control should 
take than they do about its potential in teleoperator systems. 

Although the experiments were not designed to measure the effectiveness 
of supervisory control during extended periods of manipulation, an interest- 
ing observation was made after the experiments had been completed - the man- 
ual experiments had been performed vith rest periods between each run because 
the subjects complained of fatigue and boredom, while the supervisory experi- 
ments had been unintentionally run back-to-back since fatigue and boredom 
were not noted. From thes ' observations it could be surmised that as a task 
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becoMs won involved end cosplez, boredoa end fetigue will becow increae- 
ingly important factors, tipping the scales even further in favor of supex^ 
visory control. However, experiments to validate this statement have yet to 
be performed. 


6. COHCLUSION 

Even under "ideal*^ control conditions i»upervisory control was found to 
be more efficient and effective (as determined from the task completion times 
and manipulation errors) than switch rate control, joystick rate control, and 
master^slave position control. Bilateral force~re flee ting master-slave was 
found to be slightly faster than supervisory control, but more prone to 
errors. Since the experiments were performed under "ideal** conditions, it 
can be reasonably predicted that supervisory control will show even more 
advantage when used with degraded sensor or control loops (e.g. , time delays, 
limited bandwidth, etc.), though the latter experiments remain to be done. 
In addition, an a posteriori observation of the experimental procedure 
appears to indicate that the effects of operator fatigue and boredom during 
extended periods of manipulation can be significantly reduced through super- 
visory control. 
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